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(54) IMAGE PROCESSOR 
(57)Abstract: 

PROBLEM TO BE SOLVED: To provide an image 
processor by which a user can correct and compose the 
images in a simple operation based on the photographed 
images without knowing the coefficients of the camera 
aberration, etc., and can effectively produce an image of a 
wide dynamic range from the photographed images only. 
SOLUTION: This image processor includes an image 
processing part 34 which displays the images photographed 
by the photographing equipments such as a digital camera 
31 , etc., and the image data including the photographing 
conditions, etc., at and image display part as the 
uncorrected and corrected images, corrects the aberration, 
color tones, etc., of the images photographed by a camera 
that has not been used before while looking at the 

uncorrected and corrected images to compose the image and to store the parameter used for 
correction in relation to the photographing equipments, and corrects the images photographed by 
a camera that has been once used before by reading out the stored parameter and composes the 
image, a monitor 7 which displays the composed images and the image data, and a printer 8 
which prints these images and image data and then outputs them to a recording medium 9. 
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* NOTICES * 

Japan Patent Office is not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may 

not reflect the original precisely. 

2 **** shows the word which can not be translated. 

3. In the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim 1] An image processing system characterized by providing the following An 
image input means input as two or more images to have an overlap field where one 
composition is divided and the same photographic subject image exists in an adjoining 
part mutually An amendment parameter setup means to set up an amendment 
parameter required to amend distortion of an image of two or more of said images 
produced in each overlap field at least, or a difference in an image An image 
amendment means which amends to an image of at least one sheet among said two or 
more images according to said set-up amendment parameter so that an image 
distortion of two or more of said images produced in an overlap field at least or a 
difference in an image may be lost Two or more images inputted by an image 
composition means to restore said one composition by connecting in piles two or 
more images amended with said image amendment means one by one on said overlap 
field, and said image input means, or an image display means to display an image of at 
least one sheet among images amended by said image amendment means 
[Claim 2] An image processing system characterized by providing the following An 
image input means to input as two or more images which photoed one composition by 
different exposure An amendment parameter setup means to set up an amendment 
parameter required to amend brightness of an image of at least one sheet among two 
or more images with which said exposure differs An image amendment means to 
amend brightness to an image of at least one sheet among said two or more images 
according to said set-up amendment parameter An image amendment means to 
display an image of at least one sheet among images amended by said image 
amendment means, Based on two or more of said input images and said set-up 
amendment parameters, the amount of incident light when said one composition is 
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inputted with said image input means is presumed. An image composition means to 
change and to compound two or more images by which brightness was amended with 
said image amendment means so that it may fall within the range [ said image display 
means ] which can be displayed 

[Claim 3] It is the image processing system of the publication of either claim 1 which 
has an amendment parameter storage means associate and memorize said 
amendment parameter in said amended image, and the identifier of the photography 
means which photoed the amendment image or its image, and carries out [ said 
amendment parameter setup means choosing a desired amendment parameter from 
the amendment parameter memorized by said amendment parameter storage means, 
and setting up, and ] as the feature, or claim 2. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[The technical field to which invention belongs] This invention relates to the image 
processing system which the image processing system which compounds two or more 
images which divided and photoed the photographic subject to plurality, or two or 
more images which were photoed by different exposure is started, especially the wide 
angle image and brightness by composition are changed, and carries out image 
composition. 
[0002] 

[Description of the Prior Art] In recent years, with improvement in a manufacturing 
technology, high-performance-izing and low-pricing progress, and a personal 
computer (PC is called hereafter) is spreading widely at a company, education, a home, 
etc. 

[0003] As image incorporation equipment to such a PC, from the silver halide film 
photoed with the conventional camera, optically, the image was changed into ejection 
and a picture signal and is captured. 

[0004] Moreover, image information machines and equipment, such as a video camera 
which photos the image other than a camera as a video signal, are used in various 
scenes. Since especially a digital still camera (a digital camera is called hereafter) 
does not use a silver halide film, but photos and records an image with a digital signal, 
it is captured as a direct image with the digital signal, and it can input it into equipment 
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and is outputted to a display or a printer, a development is unnecessary for it and 
elimination and edit are easy for it. Since the increment in the Internet user and the 
low price of the digital camera itself progressed in addition to these reasons, the need 
of a digital camera has spread quickly focusing on PC user. 

[0005] However, by solid state image sensors, such as CCD for which this digital 
camera used photo electric conversion, although a photographic subject image is 
picturized as a picture signal by photo electric conversion, since resolution and a 
dynamic range are inferior as compared with a silver halide film, technology of 
high-resolution-izing and the formation of an extensive dynamic range is desired 
strongly. 

[0006] As one of the method of this of high-resolution-izing, how to increase the 
number of pixels of an image sensor can be considered. However, generally it is known 
that the cost of a solid state image sensor will go up quickly according to increase of 
the number of pixels. 

[0007] Then, these people did division photography of the photographic subject by 
migration of the technology which compounds the image photoed with two or more 
image sensors which are indicated by JP,6-141246,A, and a camera which is indicated 
by JP,6-141228,A etc., and have proposed the technology which compounds two or 
more images obtained by one image sensor. 

[0008] However, since the image is distorted in response to the effect of the 
distortion aberration by optical system, when it compounds with technology which is 
indicated by the official report mentioned above, duplication portions differ by right 
and left, and the problem from which the compounded photographic subject becomes 
a duplex produces the usually photoed image. Moreover, although migration of the 
location of some points used as the criteria of superposition becomes a cause and the 
image is not rotating in fact, it will be detected as what rotated and the problem that 
composition is lost in a superior pile arises. 

[0009] Therefore, these people proposed the image processing system which 
compensates the effect of aberration with establishing in a processor an image 
amendment means to perform geometric amendment in JP.8-1 16490,A. The example 
of a configuration of this image processing system is shown in drawing 18 . 
[0010] The image input sections 1a-1c of this image processing system are the image 
pick-up sections 3, such as optical system 2 and CCD, and A/D. It consists of 
transducers 4, a duplication field is given for the portion (location) from which a 
photographic subject 5 differs, respectively, and it is arranged so that it may picturize. 
[001 1] The output signal of this image pick-up section 3 is digitized by the 
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A/D-conversion section 4, and is inputted into the image amendment sections 
17a-17c. Said image amendment sections 17a-17c read the property parameter of 
photography conditions, such as a focal location further recorded at the time of 
photography, or optical system, and are the image input section. The distortion 
aberration of the image photoed by 1a-1c etc. is amended. 

[0012] Next, in the image composition section 6, it is the technology which makes an 
input signal the image amended in the image amendment sections 17a-17c, and 
mentions it later, and it is compounded by the wide angle image as shown in drawing 
20 , and is outputted to a monitor 7, a printer 8, and storage 9 grade. 
[0013] Said image composition section 6 is realized by the configuration as shown in 
drawing 19 . In this configuration, each images a, b, and c are temporarily memorized 
by the frame memory 10, respectively, and the amount S1 of parallel displacements 
between adjacent images (for example, Image a and an image (b)) and a rotation R1 
shift, and they are called for by detector 11a. Similarly, the amount S2 of parallel 
displacements between Image b and Image c and a rotation R2 shift, and it asks by 
detector 11b. 

[0014] These amounts S1 and S2 of parallel displacements and rotations R1 and R2 
are inputted into the interpolation computing elements 1 2a and 1 2b, respectively with 
the image by which reading appearance was carried out from frame memories 10b and 
10c, and the image which amended physical relationship is created. 
[0015] Moreover, in the coefficient setter 13, the coefficients C a, Cb, and Cc of each 
image as shown in drawing 20 are set up so that an adjoining image may be connected 
smoothly. Coefficients C a, Cb, and Cc can take advantaging of the pixel value of each 
image in the multiplication section 14, and the overlapping portion is added to it by the 
adder unit 15. 

[0016] Drawing 20 is drawing for explaining processing of the lap portion of the image 
compounded. Image b is rotating counterclockwise to Image a. It shifts and this 
rotation and the amount of overlap (the amount of parallel displacements) are 
computed with a detector 1 1 . Moreover, the portion which Images a and b overlap 
carries out the multiplication of the coefficients C a, Cb, and Cc as it indicated to 
drawing 20 that it is smoothly combined, and adds them, thus, high resolution - which 
compounded two or more images — extensive — a field angle image is outputted from 
the image composition section 6. 

[0017] Moreover, about the formation of an extensive dynamic range of an image 
sensor, these people compound two or more images which changed and photoed 
exposure by JP,63-232591 A and are indicating the technology which generates an 
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image with about the same dynamic range as a silver halide film. 

[0018] This technology is realizable with constituting the image composition section 6, 
as shown in drawing 22 . By drawing 22 , since it is easy, the example which 
compounds the image of two sheets is explained notionally, but even if it is 
composition of three or more sheets, it is compoundable with the same processing. 
[0019] The images a and b of two sheets are added with an adder 21, and are stored in 
a frame memory 10. The linear transformation section 22 reads the data of a frame 
memory 10, computes the value corresponding to R of the incident light at the time of 
photography, G, and B value based on a translation table, and inputs it into the matrix 
circuit 23. At this time R, G, and B value are an image beyond the die NAMMIKU range 
of input devices, such as a digital camera. 

[0020] moreover, a translation table — the translation table creation section 27 — it 
is — exposure-time ratio Rexp of the image of two sheets from — it is decided. In the 
matrix circuit 23, the luminance-signal value Y is calculated from R, G, and B value, 
from the luminance-signal compression zone 24, brightness value Y' which 
compressed gradation according to the output unit outputs — having — a divider 25 
— ratio Y' with the original brightness value Y — /Y is called for. this ratio — Y7Y — 
output of the linear transformation section 22 Multiplication is carried out to R, G, and 
B with a multiplier 25, and it is stored in a frame memory 16 as a synthetic image 
result. 

[0021] By the way, if the signal which an image sensor generally outputs is made into 
the example of prolonged exposure as shown in drawing 21 , it will be saturated above 
a certain amount of incident light depending on exposure, and will become fixed. Then, 
the signal adding the signal of prolonged exposure and short-time exposure changes 
to the amount of incident light like the polygonal line indicated to be an addition signal 
by drawing 21 . So, in the translation table creation section 27, a table which presumes 
the amount I of incident light from a certain addition signal value S is created. 
[0022] moreover — general — an image value — 0 - in order for 256 gradation of 255 
to express, at the brightness compression zone 24, the brightness Y of each pixel is 
compressed for example, by (1) formula. 
Y'=b-Ya (1) 

a is the coefficient which determines a compressive configuration here, and b is a 
coefficient which determines the gain of the whole image. 
[0023] If the image of two sheets with which the exposure time is different is 
compounded by the above methods, the image which had about the same dynamic 
range as a silver halide film, and had good exposure from the dark place to the bright 
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place can be obtained. 
[0024] 

[Problem(s) to be Solved by the Invention] However, in the Prior art indicated by 
JP,8-116490,A mentioned above, since it cannot set up if the parameter of 
photography conditions required for distortion aberration amendment or optical 
system is not known beforehand, it is difficult to create high resolving, an extensive 
field angle image, and a panorama image simply using the motion picture camera 
material of the arbitration which a user owns. 

[0025] The example of distortion aberration is shown in drawing 23 . Distortion 
aberration is geometrical deformation generally produced according to the distance 
from the center of a lens. When it lets optical system without distortion aberration 
pass, for example, a grid-like photographic subject is photoed, it comes to be shown in 
drawing 23 (a). However, if optical system has distortion aberration, as shown in 
drawing 23 (b), the portion which should be reflected in a straight line essentially will 
bend, and it will be reflected, for example. Thus, when distortion aberration is in optical 
system, as shown in drawing 23 (c), a straight line L is distorted like curvilinear L\ and 
in order to carry out image formation, it moves the point Y on a straight line L to 
curvilinear U up point Y\ The amount [ in / here / certain one on an image ] DR of 
aberration deltaR=A1, R3+A2, and R5+— (2) 
** — it is expressed with a polynomial [ like ]. 

[0026] Here, R is the distance from an image center to Point Y. In order to create the 
image which made aberration amend, only amount of aberration deltaR should move in 
the straight line top which connects point Y' for point Y' to the image center C. 
However, it is difficult for a regular user for a coefficient A 1 , A2, and — to change 
with focal locations of optical system, and to get to know a coefficient A 1 , A2, and — 
correctly. 

[0027] Moreover, since a coefficient A 1, A2, and — change also with equipments to 
be used, when using different equipments, a correction factor must be adjusted anew. 
Furthermore, if a zoom ratio is changed and photoed, magnitude differs between the 
adjacent images, and in spite of overlapping mutually and taking a photograph, 
lamination ****** will not be made correctly. 

[0028] Moreover, even when photoing the photographic subject of near, such as the 
interior of a room, and a photography person moves several steps, magnitude will 
change, moreover — the case where they are the equipments with which a white 
balance is adjusted automatically — for example, When the direction near the sun was 
photoed, and when the direction distant from the sun is photoed, the case where tints 
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differ greatly can be considered. Therefore, although a tint changes smoothly when an 
image is compounded with the aforementioned technology, it will become an image 
unnatural as a whole. Moreover, the point N that it is necessary in the technology of 
the formation of extensive dynamic to create the table which presumes the amount of 
incident light from an addition signal but as the conventional technology explained, and 
the inclination of an addition signal changes as shown in drawing 21 is the 
exposure-time ratio Rexp. It changes. Therefore, exposure-time ratio Rexp of two or 
more images for composition To know beforehand is needed. However, it is even if it 
can perform exposure adjustment in the digital camera marketed, It has the 
composition that a user cannot know the ratio, in many cases. 
[0029] Then, it aims at offering the image processing system which can create 
[ offering the image processing system which amends by easy actuation and can be 
compounded based on the image photoed in this invention even if it did not know 
coefficients, such as aberration of a camera, and ] an extensive dynamic range-ized 
image effectively only from the photoed image. 
[0030] 

[Means for Solving the Problem] An image input means input as two or more images to 
have an overlap field where the same photographic subject image exists in a part 
which divides one composition in order that this invention may attain the 
above-mentioned purpose, and adjoins mutually, An amendment parameter setup 
means to set up an amendment parameter required to amend distortion of an image of 
two or more of said images produced in each overlap field at least, or a difference in 
an image, According to said set-up amendment parameter, so that an image distortion 
of two or more of said images produced in an overlap field at least or a difference in an 
image may be lost Two or more images amended with an image amendment means 
which amends to an image of at least one sheet among said two or more images, and 
said image amendment means by connecting one by one in piles on said overlap field 
An image processing system equipped with two or more images inputted by image 
composition means to restore said one composition, and said image input means, or an 
image display means to display an image of at least one sheet among images amended 
by said image amendment means is offered. 

[0031] Since an image processing system of the above configurations displays an 
image amended with an image amendment means on an image display means and can 
check it, even if it does not know a required amendment parameter, an image can be 
amended effectively, and an image correctly connected with an image composition 
means is obtained. 
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[0032] Moreover, an image input means to input as two or more images which photoed 
one composition by different exposure, An amendment parameter setup means to set 
up an amendment parameter required to amend brightness of an image of at least one 
sheet among two or more images with which said exposure differs, An image 
amendment means to amend brightness to an image of at least one sheet among said 
two or more images according to said set-up amendment parameter, An image 
amendment means to display an image of at least one sheet among images amended 
by said image amendment means, Based on two or more of said input images and said 
set-up amendment parameters, the amount of incident light when said one 
composition is inputted with said image input means is presumed. An image processing 
system equipped with an image composition means to change and to compound two or 
more images by which brightness was amended with said image amendment means so 
that it may fall within the range [ said image display means ] which can be displayed is 
offered. 

[0033] In such an image processing system, an image amended with an image 
amendment means is displayed on an image display means, an amendment parameter 
is set up, and an image by which brightness was amended with an image composition 
means so that it might fit in the range of an image display means which can be 
displayed based on this amendment parameter is obtained. 

[0034] Said image processing system has an amendment parameter storage means 
associate and memorize said amendment parameter in said amended image, and an 
identifier of a photography means which photoed the amendment image or its image, 
and said amendment parameter setup means provides the image processing system 
which chooses and sets up a desired amendment parameter from an amendment 
parameter memorized by said amendment parameter storage means. 
[0035] Furthermore, an image processing system saves parameter value once used 
for amendment for an amendment parameter storage means, is choosing a required 
value from an amendment parameter storage means, and using it, and does not need 
to amend it from the start anew for every image. 
[0036] 

[Embodiment of the Invention] Hereafter, the operation gestalt of this invention is 
explained to details with reference to a drawing. The rough configuration of the image 
processing system as 1st operation gestalt by this invention is shown in drawing 1 , 
and it explains to it. Looking at the display of a display etc., simple, aberration is 
amended and this operation gestalt is equipment which compounds the amended 
image and obtains the image of high resolution and an extensive field angle. 
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[0037] The memory card 32 which records the image data which this image processing 
system becomes from photography information, such as an image photoed by the 
digital still camera (a digital camera is called hereafter) 31, and photography conditions 
at the time of photography, The card reader 33 which reads image data in a memory 
card 32, and the image-processing section 34 which reproduces an image from the 
read image data, amends aberration, a color tone, etc. in those images, and is 
compounded, It consists of record media 9, such as an optical disk, a memory card, 
etc. which memorize the monitor 7 which displays the image and image data which 
were compounded, the printer 8 which carries out a printout, an image, etc. 
[0038] Said image-processing section 34 consists of the image data playback section 
35 which processes expanding etc. from the image data read by the card reader 33, 
and reproduces an image and photography information, the image amendment 
processing section 36 which amends aberration, a color tone, etc. in an image, the 
signal change section 37 which performs the image change for image composition, and 
the image composition section 6 which compounds an image. 
[0039] The image which the user divided by such configuration so that some 
photographic subject images might overlap mutually using a digital camera 31 is 
photoed. These photoed images (Image a, Image b, — ) are recorded on a memory card 
32 as image data, after processing of addition of compression and header information 
etc. is performed within a digital camera 31. This memory card 32 is inserted in a card 
reader 33, and reading appearance of the recorded image data is carried out from a 
memory card 32, and it is incorporated by the image-processing section 34. First, it is 
inputted into the image data playback section 35, processing of expanding etc. is 
performed, and the incorporated image data is reproduced as image data. 
[0040] And amendment processing is performed in the image amendment processing 
section 36, and the reproduced image data passes along the signal change section 37, 
and is inputted into the image composition section 6. The image composition section 6 
shall be equipped with the configuration shown in drawing 19 . This image composition 
section 6 performs and compounds the same processing as JP,6-141228,A, and 
outputs that synthetic image to a monitor 7, a printer 8, and storage 9 grade. 
[0041] Next, the detailed configuration of the image amendment processing section 36 
mentioned above is shown in drawing 2 , and it explains to it. The image display section 
44 as which this image amendment processing section 36 displays a subject-copy 
image and an amendment image, The aberration amendment processing section 41 
which performs aberration amendment to the inputted image, and the amendment 
parameter storage section 42 which memorizes the parameter used for amendment in 
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this aberration amendment processing section 41, It consists of the amendment 
parameter setup sections 43 which choose and set up the amendment parameter by 
which adjusted the amendment parameter by actuation of a user and reading 
appearance was carried out from a setup or the amendment parameter storage 
section 42. 

[0042] Said image amendment processing section 36 serves that is different by the 
case where aberration amendment is performed for the first time to the image which 
had never performed aberration amendment to the photoed image in the camera 
which takes a photograph, and was photoed with the camera, and the case where the 
image photoed with the camera with which the photography image has been amended 
before and the aberration amendment history remains is amended. 
[0043] First, the case where aberration amendment is carried out is explained to the 
image photoed with the camera used for the first time. In said aberration amendment 
processing section 41, an image (for example, the image a) with distortion aberration 
as shown in drawing 23 (b) is inputted, and the image of the coefficient A 1 in (2) types 
set up beforehand, A2, and — amended according to (2) types is outputted to the 
image display section 44 using initial value. This image display section 44 displays the 
subject-copy image a on an amendment image and coincidence. 

[0044] And in the amendment parameter setup section 43, if it has the device in which 
a coefficient A 1, A2, and — are adjusted and a desired coefficient is set up by 
actuation of a user, the new coefficient A 1, A2, and — will be fed back to the 
aberration amendment processing section 41 so that a coefficient may be updated 
immediately. The aberration amendment processing section 41 outputs the image 
amended by the coefficient after newly setting up, and the display of the image display 
section 44 is updated by the newly amended image. 

[0045] Looking at the image of two sheets or at least one sheet, a user operates the 
amendment parameter setup section 43, and determines the coefficient A 1 which 
performs optimal amendment to the image from the camera used for photography, A2, 
and — . When it constitutes so that the imagination adjustment tongue displayed on 
the image display section 44 may be operated by the mouse, a keyboard, etc. as the 
amendment parameter setup section 43 was shown in drawing 3 (a) at this time, 
actuation is simple and desirable. If this tongue is operated and a coefficient ( drawing 
3 A1, A 2) is changed, "the image after amendment" on a screen will be amended and 
updated according to a new coefficient value. Of course, when two or more sheets 
need to be displayed, the image of the number of sheets of arbitration may be 
displayed on the display screen. 
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[0046] And when it is judged that amendment is enough, a user presses the "O.K." 
key (clicking in the case of a mouse), and determines a coefficient A 1, A2, and — . 
Thus, the determined coefficient is memorized by the amendment parameter storage 
section 42 with the identifier users, such as a model name of a camera, named 
arbitration, respectively. What is necessary is just to make the amendment parameter 
storage section 42 memorize this as a file of format as shown in drawing 4 . 
[0047] Although the image used for the coefficient adjustment mentioned above is the 
thing of arbitration and is not cared about, the image with which the straight line which 
intersects perpendicularly photoed the photographic subject included like a building, a 
bookshelf, a window frame, or section paper tends to carry out [ adjustment ] it. 
[ many ] 

[0048] Next, the case where the image photoed with the camera with which the 
photography image had been amended in the past, and the coefficient A 1 at that time, 
A2, and — are saved as a file in the amendment parameter storage section 42 is 
amended is explained. 

[0049] In such a case, in case an image processing system is started, the data is read, 
and since an amendment parameter is related with the identifier which shows motion 
picture camera material and is memorized, as shown in drawing 3 (b), it displays on the 
image display section 44 in menu format. And reading appearance of the coefficient A 
1 to which he corresponds if a user chooses a use equipments name from the menu 
screen, A2, and — is carried out through the amendment parameter setup section 43 
to the aberration amendment processing section 41 from the amendment parameter 
storage section 42, and the image which performed aberration amendment processing 
to the input image is outputted from the aberration amendment processing section 41. 
[0050] It is not necessary to know the aberration information on the optical system of 
motion picture camera material beforehand, and with this operation gestalt, as 
explained above, in order to determine a correction factor, looking at only the image 
which the user photoed, even if it is the camera which was not being used until now, it 
faces using it and it is not necessary to know the property of the optical system in 
advance. 

[0051] Moreover, since the correction factor of aberration is once recorded on the 
photoed image as data to the camera which had performed aberration amendment and 
amendment processing can be carried out only by choosing from a menu, the 
complicatedness of actuation in which a parameter must be set up and amended to all 
the images used for image composition each time is avoidable. 

[0052] In addition, although the digital camera which memorizes an image is used for 
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the memory card with this operation gestalt, it is not limited to this but an image 
processing is possible also for equipments which are inputted into the image 
reconstruction section directly. Moreover, although the case where two or more 
images by one set of an input unit are photoed is explained, also in the configuration 
which uses and inputs two or more input units into coincidence, it is possible. 
Moreover, this operation gestalt is available for it also as a switch of rotating type and 
slide-type another object, although the amendment parameter setup section 43 was 
explained as an imagination adjustment tongue shown on the image display section 44. 
[0053] Furthermore, although it is the configuration which is amend while compare a 
subject copy image and a processing result with this operation gestalt, if the amount 
of bays, such as a building and a bookshelf, are the images by the aberration [ have ] 
to include [ many ], a coefficient A 1 , A2, and — can be determine by display only the 
image after amendment processing on the image display section 44, and adjust so that 
the amount of bay may become straight. 

[0054] Next, the image processing system as 2nd operation gestalt is explained. This 
operation gestalt is deformation of the 1st operation gestalt mentioned above, and is 
explained with reference to drawing 5 thru/or drawing 8 . By the configuration part of 
this operation gestalt, the same reference mark is given to a part equivalent to the 
configuration part shown in drawing 1 and drawing 2 which were mentioned above, and 
the explanation is omitted to it. 

[0055] Drawing 6 is drawing showing the example of a configuration of this operation 
gestalt, and it differs from the 1st operation gestalt in that the synthetic image 
outputted from the image composition section 6 is inputted into the image amendment 
processing section 36. Drawing 7 shows the example of a configuration of the image 
amendment processing section 36. Drawing 8 shows an example of the screen 
displayed on the image display section 44 of the image amendment processing section 
36. 

[0056] In this configuration, the image data (here the image a) to which expanding etc. 
was given in the image data playback section 35 is inputted into the aberration 
amendment processing section 41, and the data amended using the amendment 
parameter value A1 set up beforehand, A2, and — is outputted to the image 
composition section 6. Also in the image which similarly adjoins the image by which 
amendment processing was carried out, it is outputted after being amended in the 
aberration amendment processing section 41, and image composition is performed in 
the image composition section 6, and the connected image is generated. And as a 
result of being compounded, an image is inputted into the image amendment 
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processing section 36, and is displayed in the image display section 44. 
[0057] Next, with reference to drawing 5 , the actual example of processing by 
actuation of a user is explained. Drawing 5 (a) is a photographic subject used as the 
candidate for photography. If through division is carried out and the optical system 
which has distortion aberration in this is incorporated, as shown in drawing 5 (b), a 
photographic subject will be distorted around an image. 

[0058] for example, the group ( drawing 5 — **** — P — one — P — one — 9 — P 

— two — P — two — f — P — three — P — three — ' — three — a set) of two or 
more focus is set up on an image. Control by the program defined beforehand may 
perform this setup automatically in the image composition section 6, and it does not 
matter even if it specifies by actuation of a user. Two points are chosen from these 
groups. Here, the example which chose P1 and P2 is explained. 

[0059] while it has not been exact, when the aberration correction factor A1 f A2, and 

— compound in quest of the amount of parallel displacements, and a rotation in the 
image composition section 6 on the basis of P1 and P2 at this time, it is shown in 
drawing 5 (c) — as — Although P1, P1\ and P2 and P2' are correctly in agreement, P3 
and P3' are not in agreement. Naturally, other surrounding each point is not in 
agreement. 

[0060] Then, it is a coefficient so that it displays the carried-out synthetic result on 
the image display section 44 by the equipment configuration shown in drawing 6 , and 
it may be in agreement except a reference point P1 and P2, while a user looks at 
those displays. A1, A2, and — are adjusted. The adjusted coefficient A 1, A2, and — 
are immediately inputted into the aberration amendment processing section 41, the 
newly amended image is connected by the image composition section 6, and 
regeneration of them is carried out to the image display section 44. 
[0061] this time — coefficient A — one — A — two — right — setting up — having 

— if — P — one — P — one — ' — P — two — P — two — ' — not only — P — 
three — P — three — ' — coincidence — being in agreement . By such method, it 
can amend easily also about unclear images of aberration, such as a landscape image 
and a portrait image, apparently. 

[0062] With this operation gestalt, the image displayed on the image display section 44 
is seen, and it is a coefficient. Although he is trying to adjust A1, A2, and — , gap of P3 
and P3 ? may be detected automatically, and a coefficient A 1 , A2, and — may be 
automatically amended so that the gap may become 0 or min. 
[0063] Next, the example of a configuration of the image amendment processing 
section of the image processing system as 3rd operation gestalt is shown in drawing 9 , 
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and it explains to it. By the configuration part of this operation gestalt, the same 
reference mark is given to a configuration part equivalent to the 1st and 2nd operation 
gestalt mentioned above, and the explanation is omitted to it. Generally, the amount of 
incident light has fallen off, so that the image photoed under the effect of limb 
darkening in the optical system of a camera etc. goes on the outskirts. Therefore, if it 
connects with the image synthesizer unit which mentioned the big image of limb 
darkening above, an image will become dark in the portion of an overlap field, and an 
unnatural synthetic image will be generated. 

[0064] It is the phenomenon in which the brightness of an image becomes dark as this 
limb darkening is shown in drawing 10 and the distance R from an image center 
becomes large, and signal value S' in case there is limb darkening is given by the 
polynomial like the following (3) types in approximation to the ideal signal value S. 
[0065] 

SVS=B0+B1, R+B2, and R2+— (3) 

Then, as shown in drawing 9 , the image amendment processing section 36 consists of 
the image display section 44, the limb-darkening amendment processing section 46, 
the limb-darkening amendment parameter storage section 47, and the limb-darkening 
amendment parameter setup selection section 48. 

[0066] In this limb-darkening amendment parameter setup selection section 48, a 
coefficient B 0, B1, B-2, and the imagination tongue that displays — on the image 
display section 44 are moved and adjusted, and it sets up so that brightness may 
become the same a center and around an image. 

[0067] By such configuration, it is a coefficient. If it constitutes so that B0, B1, B-2, 
and — may be set up, even if it will not know many parameters of the optical system 
of the camera used for photography, the overlap section does not become dark 
unnaturally, either but, as for the image which can amend easily based on the photoed 
image and was connected in the image composition section 6, an image natural as the 
whole screen can be obtained. 

[0068] Moreover, the value of the coefficient which once amended and was set up as 
the 1 st operation gestalt explained, since it was the phenomenon of a proper is related 
with the name of motion picture camera material etc., and is saved at the optical 
system of motion picture camera material as well as [ limb darkening ] distortion 
aberration, and 2nd henceforth can be amended only by choosing the name from a 
menu. 

[0069] Furthermore, although this operation gestalt explained that it amended while 
comparing a subject-copy image and a processing result, since the brightness of the 
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whole image should just become uniform, only the image after amendment processing 
is displayed on the image display section 44, and they are a coefficient B 0, B1, B-2, 
and — . You may make it set up and choose. 

[0070] Next, with reference to drawing 1 1 and drawing 1 2 , the configuration which 
performs color tone amendment of the image processing system as 4th operation 
gestalt is explained. The image amendment processing section 36 of a configuration of 
being shown in drawing 1 1 consists of the amendment parameter setup selection 
sections 43 which adjust the color tone amendment section 51 changed into a hue 
(Hue;H), saturation (Saturations), and lightness (Intensity;I), respectively, and the 
amendment parameter which performs conversion of H, S, and I by conversion which 
calls HSI conversion the image display section 44 and a subject-copy image, and 
amend a color tone. 

[0071] A subject-copy image is inputted into the color tone amendment section 51 by 
such the image amendment processing section 36, respectively. It is changed into Hue 
H, saturation S, and Lightness I by HSI conversion. Furthermore, hue H set up in the 
amendment parameter setup section 43 in the color tone amendment section 51 
Based on Color S and Lightness I, an input image is amended and it outputs to the 
image display section 44. A user is the hue H on a screen while comparing the image 
displayed on the image display section 44, as shown in drawing 12 . The tongue for 
adjustment of Color S and Lightness I is operated, and it adjusts so that both may 
become the same color tone. And the data with which the color tone was adjusted is 
compounded and connected in the image composition section 6. 
[0072] Thus, the color tone of not only the overlap field of the image which 
compounded two or more images which adjoin each other in the case of image 
composition by the constituted image amendment processing section 36 but the 
whole screen will be able to become natural, of course, doubling a white balance, 
although this operation gestalt explained that it changed and adjusted to a hue, color, 
and lightness — the same — R, G, and B — you may make it double each signal level 
[0073] As furthermore shown in drawing 13 , also corresponding to the difference in a 
zoom ratio, and magnitude change of the photographic subject by a photography 
person's camera station, a good synthetic image can be obtained by equipping the 
image amendment processing section 36 with the image enlarging-or-contracting 
section 52 and the amendment parameter setup selection section 43. 
[0074] Next, with reference to drawing 14 thru/or drawing 1 7 , the example of a 
configuration which performs extensive dynamic-ized processing by the image 
processing system as 5th operation gestalt is shown, and it explains. This image 
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processing system is the exposure-time ratio Rexp from the image already photoed in 
extensive dynamicHzed technology. It is equipment which computes and compounds 
an image using this exposure-time ratio. The same reference mark is given to a part 
equivalent to the configuration part of the 1st operation gestalt mentioned above by 
the configuration part of this operation gestalt, and the explanation is omitted. 
[0075] The image processing system of this operation gestalt is the configuration 
which equipped the image-processing section 34 with the exposure-time ratio 
calculation section 61. In this image-processing section 34, the reproduced images a 
and b are first inputted into the exposure-time ratio calculation section 61 via the 
signal change section 37, and the exposure-time ratio Rexp is inputted into the image 
composition section 6 as a result. Exposure-time ratio Rexp computed in this image 
composition section 6 It uses, Images a and b are compounded and the image beyond 
the dynamic range of an input device is outputted. In addition, since the configuration 
of the image composition section 6 uses the same configuration as drawing 22 , it 
omits explanation here. 

[0076] The example of a configuration of said exposure-time ratio calculation section 
61 is shown in drawing 1 5 , and it explains to it. This exposure-time ratio calculation 
section 61 is [ the image display section 44 and ] the exposure-time ratio Rexp by the 
user. Exposure-time ratio Rexp set up in the exposure-time ratio setting section 63 
for adjusting, and this exposure-time ratio setting section 63 It consists of the 
brightness amendment sections 62 which adjust brightness to a radical to an input 
image. 

[0077] Thus, the constituted exposure-time ratio calculation section 61 displays one 
in the input images a and b (here, it considers as Image a) on the image display section 
44 as it is as a criteria image. Moreover, Image b amends brightness in the brightness 
amendment section 62, and is outputted and displayed on the image display section 44. 
[0078] For said exposure-time ratio setting section 63, a user is the exposure-time 
ratio Rexp. It has the device which displays an imagination adjustment tongue on the 
display screen of the image display section 44 mentioned later so that it can adjust. A 
user is the exposure-time ratio Rexp. It is immediately fed back to the brightness 
amendment section 62, and the adjusted result is the exposure-time ratio Rexp after 
adjustment. The amended image is displayed in the image display section 44. 
[0079] Drawing 1 6 is drawing having shown processing in the brightness amendment 
section 62. Input picture signal Sin before amendment It receives and is the 
amendment backward signal Sout. Sout =Rexp and Sin (4) 

It is changed. At this time, if it becomes Rexp >1.0, an image will become bright. An 
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image will become dark if it becomes Rexp <1.0. 

[0080] Drawing 17 is the example of a display of the image display section 44. In the 
image a of the criteria image shown in this drawing 1 7 , the interior of a room (place 
with a table, a flower, etc.) is proper exposure, and the outside (place a tree and a 
mountain appear) of an aperture expresses the condition of being exaggerated 
exposure. Moreover, in the image b of an adjustment image, the outside of said 
aperture is proper exposure and said interior of a room expresses the condition of 
being undershirt exposure. 

[0081] Image a adjusts the tongue which is displayed as a criteria image and displayed 
on a screen, and amends the brightness of Image b. And when the brightness of the 
whole image b becomes the about the same* as a criteria image (image a), a user is the 
"O.K." key and directs adjustment termination. Thus, set-up exposure-time ratio 
Rexp It is outputted to the image composition section 6, and extensive dynamic 
range-ized processing is performed. 

[0082] It is the exposure-time ratio Rexp, seeing and checking an image according to 
this operation gestalt. Since it can ask, when the device for which it asks to the input 
device for inputting an image can be used and a photograph is taken, the necessity of 
recording the exposure time is lost, and extensive dynamic range-ized processing can 
be performed easily. 

[0083] Moreover, although the example adjusted as a parameter which amends the 
exposure-time ratio by the image of two sheets explained with this operation gestalt, 
also when it is not limited to this and the exposure-time ratio of the image of three or 
more sheets is made into an amendment parameter, of course, it is possible. 
Furthermore, although it explained that only the image of another side carried out 
brightness amendment on the basis of one image in an image, of course, the 
brightness of both images may be amended and the image of in-between brightness 
may be generated. The exposure-time ratio at this time turns into a ratio of the 
inclination shown in drawing 16 in each image. 

[0084] Furthermore, it is also possible to change and photo drawing as a method of 
photoing two or more light-and-darkness images with which exposure differs, and to 
control by the permeability of an extinction filter. That is, it is [ that what is necessary 
is just what shows the exposure ratio between two or more images as an amendment 
parameter required for extensive dynamic range-ized processing ] possible to use the 
exposure-time ratio shown in the above-mentioned operation gestalt, and a 
contraction ratio and the permeability ratio of an extinction filter as an amendment 
parameter as what shows such an exposure ratio. 
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[0085] Moreover, in all operation gestalten, although it has described that a 
photograph is taken with a digital camera, also in the image which incorporated the 
film photoed with the conventional film-based camera through the image taking-in 
board etc. from the image photoed with the image which digitized with the scanner, 
the video camera, or the digital camcorder, the same processing is possible. The 
monitor 7 which outputs the image after being tied can be made to make it serve a 
double purpose also in the image display device 44 in each operation gestalt. 
[0086] Furthermore, it is also possible to constitute combining respectively each 
operation gestalt mentioned above, especially — the amendment parameter storage 
section 42 — above-mentioned the 1- in the configuration indicated by the 3rd 
operation gestalt, if the property of optical system is known, it can also memorize, it 
can include in the 4th and 5th operation gestalt, can consider as the default of 
adjustment by memorizing a parameter, and can also be made the reference at the 
time of a user adjusting. 

[0087] Although the above operation gestalt was explained, the following invention is 
also included in this specification. 

An image input means input as two or more images to have the overlap field where 1.1 
composition is divided and the same photographic subject image exists in an adjoining 
part mutually, An amendment parameter setup means to set up an amendment 
parameter required to amend the distortion of the image of two or more of said images 
produced in each overlap field at least, or the difference in an image, According to said 
set-up amendment parameter, so that the image distortion of two or more of said 
images produced in the overlap field at least or the difference in an image may be lost 
Two or more images amended with the image amendment means which amends to the 
image of at least one sheet among said two or more images, and said image 
amendment means by connecting one by one in piles on said overlap field The image 
processing system characterized by providing two or more images inputted by image 
composition means to restore said one composition, and said image input means, or an 
image display means to display the image of at least one sheet among the images 
amended by said image amendment means. 

[0088] It corresponds to a 1st and 2 operation gestalt. Since the image amended with 
the image amendment means is displayed on an image display means and can be 
checked with this image processing system, even if it does not know a required 
amendment parameter, an image can be amended effectively, and the image correctly 
connected with the image composition means can be obtained. 
[0089] An image input means to input as two or more images which photoed 2.1 
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composition by different exposure, An amendment parameter setup means to set up 
an amendment parameter required to amend the brightness of the image of at least 
one sheet among two or more images with which said exposure differs, An image 
amendment means to amend brightness to the image of at least one sheet among said 
two or more images according to said set-up amendment parameter, An image 
amendment means to display the image of at least one sheet among the images 
amended by said image amendment means, Based on two or more of said input images 
and said set-up amendment parameters, the amount of incident light when said one 
composition is inputted with said image input means is presumed. The image 
processing system characterized by **(ing) an image composition means to change 
and to compound two or more images by which brightness was amended with said 
image amendment means so that it may fall within the range [ said image display 
means ] which can be displayed. 

[0090] It corresponds to the 5th operation gestalt. In this image processing system, 
the image amended with the image amendment means is displayed on an image display 
means, an amendment parameter is set up, and the image by which brightness was 
amended with the image composition means so that it might fit in the range of an 
image display means which can be displayed based on this amendment parameter is 
obtained. 

[0091] 3. said — an image — amendment — a means — said — image display — a 
means — displaying — having — plurality — an image — between — it can set — an 
image — brightness — a difference — responding — said — amendment — a 
parameter — changing — making — things — an image — amending — a thing — it is 

— things — the feature — ** — carrying out — the above — ( — one — ) — a term 

— or — ( — two — ) — a term — 

[0092] It corresponds to the 5th operation gestalt. In this image processing system, a 
parameter required for extensive dynamic range-ized processing can be obtained 
from an image, without recording an exposure-time ratio, a contraction ratio, etc. 
beforehand by amending brightness, checking an amendment result on an image 
display means. 

[0093] 4. said — an image — amendment — a means — said — image display — a 
means — displaying — having — one — a sheet — an image — distortion — 
aberration — or — plurality — an image — between — it can set — distortion — 
aberration — a difference — responding — said — amendment — a parameter — 
changing — making — things — an image — amending — a thing — it is — things — 
the feature — ** — carrying out — the above — ( — one — ) — an 
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[0094] It corresponds to a 1st and 2 operation gestalt. In this image processing 
system, since distortion aberration is amended checking an amendment result on an 
image display means, it can ask for gap between contiguity images, and rotation 
correctly. 

[0095] 5. said — an image — amendment — a means — said — image display — a 
means — displaying — having — plurality — an image — between — it can set — an 
image — a scale factor — a difference — responding — said — amendment — a 
parameter — changing — making — things — an image — amending — a thing — it is 

— things — the feature — ** — carrying out — the above — ( — one — ) — a term 

— a publication — an image processing system . 

[0096] It corresponds to the 4th operation gestalt. With this image processing system, 
connector doubling of an exact image becomes possible by amending the difference in 
a scale factor, checking an amendment result on an image display means. 
[0097] 6. said — an image — amendment — a means — said — image display — a 
means — displaying — having — plurality — an image — between — it can set — an 
image — a color — information — a difference — responding — said — amendment 

— a parameter — changing — making — things — an image — amending — a thing 

— it is — things — the feature — ** — carrying out — the above — ( — one — ) — 
a term — a publication — an image processing system . 

[0098] It corresponds to the 4th operation gestalt. In this image processing system, 
the image connected by amending the difference in the color between the images 
which adjoin while checking an amendment result on an image display means turns 
into a very natural image. 

[0099] 7. Said color information is an image processing system given in the 
aforementioned (6) term characterized by being at least one of a hue, saturation, and 
the lightness. It corresponds to the 4th operation gestalt. In this image processing 
system, the image connected by amending the difference in the color between the 
images which adjoin while checking an amendment result on an image display means 
turns into a very natural image. 

[0100] 8. said — a color — information — a white balance — adjusting — a sake — R 

— G — B — a value — at least — one — a ** — it is — things — the feature — ** 

— carrying out — the above — ( — six — ) — a term — a publication — an image 
processing system . It corresponds to the 4th operation gestalt. In this image 
processing system, the image connected by amending the difference in the color 
between the images which adjoin while checking an amendment result on an image 
display means turns into a very natural image. 



20/24 



Japanese Publication number : 10-187929 

[0101] 9. said — an image — amendment — a means — said — image display — a 
means — displaying — having — one — a sheet — an image — it can set — an image 

— limb darkening — or — plurality — an image — between — it can set — an image 

— limb darkening — a difference — responding — said — amendment — a parameter 

— changing — making — things — an image — amending — a thing — it is — things 

— the feature — ** — carrying out — the above — ( 

[0102] It corresponds to the 3rd operation gestalt. In this image processing system, in 
order to amend limb darkening, checking an amendment result on an image display 
means, the connected image turns into a very natural image. 

[0103] 10. said — an image — amendment — a means — said — image display — a 
means — displaying — having — plurality — an image — between — it can set — an 
image — brightness — a difference — responding — said — amendment — a 
parameter — ****** — said — plurality — an image — between — it can set — 
exposure — a ratio — changing — making — things — an image — amending — a 
thing — it is — things — the feature — ** — carrying out — the above — 
[0104] It corresponds to the 5th operation gestalt. In this image processing system, 
checking an amendment result on an image display means, by changing the exposure 
ratios between two or more images as a parameter required for extensive dynamic 
range-ized processing (an exposure-time ratio, a contraction ratio, permeability ratio, 
etc.), the brightness of an image can be amended and the synthetic image formed into 
the extensive dynamic range can be obtained. 

[0105] 1 1. It is the image processing system of the publication of the above (1) which 
has an amendment parameter storage means associate and memorize said 
amendment parameter in said amended image, and the amendment image or the 
identifier of the photography means which photoed the image, and carries out [ said 
amendment parameter setup means choosing a desired amendment parameter from 
the amendment parameter memorized by said amendment parameter storage means, 
and setting up, and ] as the feature, or (3) terms. 

[0106] It corresponds to a 1st, 2, and 3 operation gestalt. The parameter value once 
used for amendment is saved for the amendment parameter storage means, and it is 
not necessary to amend from the start anew for every image by choosing a required 
value from an amendment parameter storage means, and using it with this image 
processing system. 
[0107] 

[Effect of the Invention] As explained in full detail above, according to this invention, it 
is not necessary to know beforehand the property of optical system, such as 
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aberration and limb darkening. Moreover, it is not necessary to use the expensive 
motion picture camera material which photography conditions, such as the exposure 
time and a white balance, can control and record. The image processing system which 
can amend and can obtain the image and the image formed into an extensive die 
NAMMIKU range of the connected extensive field angle only from the image photoed 
by the motion picture camera material and the cheap motion picture camera material 
of the arbitration which a user owns can be offered. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing ll It is drawing showing the rough configuration of the image processing 
system as 1st operation gestalt by this invention. 

[Drawing 2] It is drawing showing the detailed configuration of the image amendment 
processing section shown in drawing 1 . 

[Drawing 31 It is drawing showing the processing directions to the image displayed on 
the image display section, and the amendment condition of an image. 
[Drawing 4] It is drawing showing an example of the format of the file memorized by the 
amendment parameter storage section. 

[Drawing 51 It is drawing for explaining composition of the image photoed by optical 
system with distortion aberration. 

[Drawing 61 It is drawing showing the rough configuration of the image processing 
system as 2nd operation gestalt by this invention. 

[Drawing 71 It is drawing showing the example of a configuration of the image 
amendment processing section shown in drawing 6 . 

[Drawing 81 It is drawing showing an example of the synthetic result displayed on the 
image display section of the image amendment processing section. 
[Drawing 91 It is drawing showing the example of a configuration of the image 
amendment processing section in the image processing system as 3rd operation gestalt. 
[Drawing 10] It is drawing showing the distance from the image center for explaining 
limb darkening, and the relation of a signal value. 

[Drawing 11] It is drawing showing the example of a configuration of the image 
amendment processing section in the image processing system as 4th operation gestalt. 
[Drawing 12] It is drawing showing the example of a display for performing the color 
tone assistant displayed on the image display section of the image amendment 
processing section. 
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[Drawing 131 It is drawing showing the example of a configuration of the image 
amendment processing section equipped with the image enlarging-or-contracting 
section and the amendment parameter setup section. 

[Drawing 141 It is drawing showing the rough configuration of the image processing 
system as 5th operation gestalt by this invention. 

[Drawing 151 It is drawing showing the example of a configuration of the exposure-time 
ratio calculation section shown in drawing 14 . 

[Drawing 16l It is drawing showing the relation between the input picture signal before 
the amendment in the brightness amendment section, and an amendment backward 
signal. 

[Drawing 171 It is drawing showing the example of a display of the image display 
section about brightness amendment. 

[Drawing 18] It is drawing showing the example of 1 configuration of the conventional 
image processing system. 

[Drawing 19] It is drawing showing the example of 1 configuration of the image 
composition section shown in drawing 18 . 

[Drawing 201 It is drawing for explaining the lap condition of the image in the wide 
angle image composition which connected the image. 

[Drawing 21] It is drawing showing the property of a signal of having added the signal 
of long duration exposure and short-time exposure. 

[Drawing 221 It is drawing showing the example of 1 configuration of the image 
composition section shown in drawing 18 . 

[Drawing 231 It is drawing showing an example of the property of distortion aberration. 
[Description of Notations] 

6 - Image composition section 

7 -- Monitor 

8 - Printer 

9 -- Record medium 

31 Digital still camera 

32 -- Memory card 

33 - Card reader 

34 Image -processing section 

35 - Image data playback section 

36 -- Image amendment section 

37 - Signal change section 

41 -- Aberration amendment processing section 
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42 - Amendment parameter storage section 

43 Amendment parameter setup section 

44 Image display section 



24/24 



